Generalization error of three layered learning model in bayesian estimation
نویسندگان
چکیده
In this paper, we obtain the asymptotic forms of the generalization errors for some three layered learning models in Bayesian estimation. The generalization error measures how precisely learning models can approximate true density functions which produce learning data. We use a recursive blowing up process for analyzing the Kullback function of the learning model. Then, we have the maximum pole of its zeta function which is defined by the integral of the Kullback function and an a priori probability density function. In [1, 2], it was proved that the maximum pole of the zeta function asymptotically gives the generalization error of the hierarchical learning model.
منابع مشابه
Learning efficiency of redundant neural networks in Bayesian estimation
This paper proves that the Bayesian stochastic complexity of a layered neural network is asymptotically smaller than that of a regular statistical model if it contains the true distribution. We consider a case when a three-layer perceptron with M input units, H hidden units and N output units is trained to estimate the true distribution represented by the model with H(0) hidden units and prove ...
متن کاملAlgebraic geometrical methods for hierarchical learning machines
Hierarchical learning machines such as layered perceptrons, radial basis functions, Gaussian mixtures are non-identifiable learning machines, whose Fisher information matrices are not positive definite. This fact shows that conventional statistical asymptotic theory cannot be applied to neural network learning theory, for example either the Bayesian a posteriori probability distribution does no...
متن کاملResolution of Singularities and the Generalization Error with Bayesian Estimation for Layered Neural Network
Hierarchical learning models such as layered neural networks have singular Fisher metrics, since their parameters are not identifiable. These are called non-regular learning models. The stochastic complexities of non-regular learning models in Bayesian estimation are asymptotically obtained by using poles of their zeta functions which are the integrals of their Kullback distances and their prio...
متن کاملA Surface Water Evaporation Estimation Model Using Bayesian Belief Networks with an Application to the Persian Gulf
Evaporation phenomena is a effective climate component on water resources management and has special importance in agriculture. In this paper, Bayesian belief networks (BBNs) as a non-linear modeling technique provide an evaporation estimation method under uncertainty. As a case study, we estimated the surface water evaporation of the Persian Gulf and worked with a dataset of observations ...
متن کاملA Surface Water Evaporation Estimation Model Using Bayesian Belief Networks with an Application to the Persian Gulf
Evaporation phenomena is a effective climate component on water resources management and has special importance in agriculture. In this paper, Bayesian belief networks (BBNs) as a non-linear modeling technique provide an evaporation estimation method under uncertainty. As a case study, we estimated the surface water evaporation of the Persian Gulf and worked with a dataset of observations ...
متن کامل